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Abstract — This paper investigates the privacy amplification 
problem, and compares the existing two bounds: the exponential 
bound derived by one of the authors and the min-entropy bound 
derived by Renner. It turns out that the exponential bound is 
better than the min-entropy bound when a security parameter 
is rather small for a block length, and that the min-entropy 
bound is better than the exponential bound when a security 
parameter is rather large for a block length. Furthermore, we 
present another bound that interpolates the exponential bound 
and the min-entropy bound by a hybrid use of the Renyi entropy 
and the inf-spectral entropy. 

I. Introduction 

The privacy amplification HI is a technique to distill a secret 
key from a source that is partially known to an eavesdropper, 
usually referred to as Eve. The privacy amplification is re- 
garded as an indispensable tool in the information theoretic 
security, and it has been studied in many literatures (eg. 0, 

ei, a, 0, m, 0, 0). 

Recently, the non-asymptotic analysis of coding problems 
has attracted considerable attention |9|, 1 10 1. Especially for the 
channel coding problem, the relation between various types of 
non-asymptotic bounds are extensively compared in ifTOl . 

The performance of the privacy amplification is typically 
characterized by the smooth minimum entropy or the inf- 
spectral entropy 0, ifTTI . 0. There is also another approach, 
the exponential bound, which has been investigated by one of 
the authors 0. So far, the relation between non-asymptotic 
bounds derived by these two approaches has not been clarified. 
The first purpose of this paper is to compare the min-entropy 
bound, which is derived by the smooth minimum entropy 
framework, and the exponential bound. Actually, it turns out 
that the exponential bound is better than the min-entropy 
bound when a security parameter is rather small for a block 
length. In the following, we explain a reason for this result. 

In the achievability part of the smooth entropy framework 
or the information spectrum approach lfl2l . a performance 
criterion of a problem, such as an error probability or a security 
parameter, is usually upper bounded by a formula consisting of 
two terms. One of the terms is caused by the smoothing error, 
which corresponds to a tail probability of atypical outcomes. 
The other is caused by typical outcomes. In the following, let 
us call the former one the type 1 error term and the latter one 
the type 2 error term respectively. 

To derive a tight bound in total, we need to tightly bound 
both the type 1 and type 2 error terms. This fact has been 
recognized in literatures. Indeed, one of the authors derived 



the state-of-the-art error exponent for the classical-quantum 
channel coding by tightly bounding both types of error terms 
lfl3l . In [ 1 1 , Polyanskiy et. al. derived a non-asymptotic 
bound of the channel coding, which is called the DT bound, 
by tightly bounding both types of error terms. The DT bound 
remarkably improves on the so-called Feinstein bound because 
the type 2 error term is loosely bounded in the Feinstein bound. 
The improvement is especially remarkable when a required 
error probability is rather small for a block length. 

For the privacy amplification problem, one of the authors 
derived the state-of-the-art exponent of the variational distance 
by tightly bounding both types of error terms [7 |. On the other 
hand, the type 2 error term is loosely bounded in the bound 
derived via the smooth minimum entropy 0. 

As is expected from the above argument, the exponential 
bound turns out to be better than the min-entropy bound when 
a security parameter is rather small for a block length. For 
rather large security parameters, the min-entropy bound is 
better than the exponential bound. This is because we derive 
the exponential bound by using the large deviation technique 
lfl4l . The large deviation technique is only tight when a 
threshold of a tail probability is away from the average, and 
this is not the case when a security parameter is rather large 
for a block length. As the second purpose of this paper, we 
derive a bound that interpolates the exponential bound and 
the min-entropy bound. This is done by a hybrid use of the 
Renyi entropy and the inf-spectral entropy. It turns out that the 
hybrid bound is better than both the exponential bound and the 
min-entropy bound for whole ranges of security parameters. 

The rest of the paper is organized as follows. In Section HU 
we summarize known bounds on the privacy amplification. 
In Section [HI] we propose a novel bound by using the 
Renyi entropy and the inf-spectral entropy. In Section [IV] we 
compare the bounds numerically. 

II. Preliminaries 

In this section, we review the problem setting and known 
results on the privacy amplification. Although most of results 
in this section were stated explicitly or implicitly in literatures, 
we restate them for reader's convenience. Especially, Theorem 
Q] Theorem |2] and Theorem [3] are classical analogue of those 
obtained in [ 8 1 for the quantum setting, where the distance to 
evaluate the smoothing is different. 



A. Problem Formulation 

For a set A, let V(A) be the set of all probability distribution 
on A. It is also convenient to introduce the set P(A) of all 
sub-normalized non-negative functions. 

Let Pxz G P(X x Z) be a sub-normalized non-negative 
function. For a function / : X — > S and the key S — f(X), 
let 

Psz{s,z)= ^2 p xz(x,z). 

xef-Hs) 

We define the security by 

d(f\P xz )=d(P S z,P s x Pz), 
where Pg is the uniform distribution on S and 



where 



d(P,Q) :=lY,\P(a)-Q(a)\ 



(1) 



for P, Q G V{A). 

Although the quantity d(f\Pxz) has no operational mean- 
ing for unnormalized Pxz, it will be used to derive bounds 
on d(f\Pxz) for normalized Pxz- For distribution Pxz G 
V{X x Z) and security parameter e > 0, we are interested in 
characterizing 

i{Pxz,£) = sup{log|5| : 3f : X -> 5 s.t. d(/|Pxz) < e}. 

B. M/n Entropy Framework 

In this section, we review the smooth minimum entropy 
framework that was mainly introduced and developed by 
Renner and his collaborators J2), 151, Ifl5ll, IfT^I, PI. 

Definition 1: For Pxz G P(A'x2) and a normalized i?^ G 
V{Z), we define 

cj- / | D n i P xz (x,z) 
Fl m in{Pxz\nz) = - log max — . 

xeA; li.y\Z ) 



Then, we define 

(Ptz|Pz) 



max #min(Qxz|-Rz), 



where 

S £ (Pxz) = {Qxz e P(X x Z) : d(P xz ,Qxz) < s} • 
We also define 

HLin( p xz\Rz) = max H min (Q X z\Rz), 

Qxz^B'(Pxz) 

where 

£ £ (Pxz) - {Qxz G V(X x Z) : d(P X z, Qxz) < e} . 

The following is a key lemma to derive every lower bound 
on £(Pxz,e). 

Lemma 1 (Leftover Hash: [2]): Let F be the uniform ran- 
dom variable on a set of universal 2 hash family T. Then, for 
Pxz G V(X x Z) and R z G V(2), we haveQ 



E F [d(F\P xz )} < -^/\S\e- H *( p x*\ R z), 

'Technically, Rz must be such that supp(Pz) C supp(iJz). 



H 2 (P XZ \Rz) = -log ^ 



sesupp(i? z ) 



is the conditional Renyi entropy of order 2 relative to Rz- 

Since H 2 {P XZ \Rz) > H min (P xz \R Z ), we have the fol- 
lowing. 

Corollary 1: For P XZ € x Z) and i? z G P(Z), we 

have 



E F [d(F|P X z)] < i^|5|e-«»i-(^z|iiz). 
Furthermore, since 

d(Pxz\.f)<2e + d(Pxz\f) 

holds for Pxz G B £ (Pxz) by the triangular inequality, we 
have the following. 

Corollary 2: For Pxz G P(A? x Z) and R z G P(Z), we 
have 



1 



E F [d(F\P xz )] < 2e+-^l\S\e- H ^ p *z\ R z). 

The following is a key lemma to derive a upper bound on 

i{Pxz,e). 

Lemma 2 (Monotonicity): For any function / : X — > <S, 
Pxz G x Z), and P z G P(Z), we have 

-ffminC-Pszl-Rz) < H^ in (P X z\Rz)- 

Proof: See Appendix lAl ■ 
Remark 1: When Eve's side-information is the quantum 
density operator instead of the random variable, the mono- 
tonicity of the smooth minimum entropy was proved in JS] 
Proposition 3], where the smoothing is evaluated by the so- 
called purified distance instead of the trace distance. For 
the quantum setting and the trace distance, it is not clear 
whether the monotonicity holds or not because we cannot 
apply Uhlmann's theorem to the trace distance directly. 

From Corollary |2] and Lemma [2] we get the following lower 
and upper bounds on £(Pxz,z)- 

Theorem 1: For any < rj < e, we have 



max H 



(Pxz\Rz) +\og4 V 2 -I 



Rzev(z) 

< £(Pxz,e) 

< H^ n (P xz \Pz). 

C. Information Spectrum Approach 

In this section, we introduce the inf-spectral entropy. The 
quantity is used to calculate the lower and upper bounds in 
Theorem Q] 

Definition 2: For Pxz G V{X X Z) and < e < 1, let 

H e s (P XZ \Rz) 

/ D f i Pxz(x,z) 
:= sup i r : P X z < - log — - ; ; < r > < e 



Rz(z) 

be the conditional inf-spectral entropy relative to Rz G V(Z). 
The following two lemmas relate the quantities 

H^ in (P xz \Rz) and H°(P X z\Rz). 



Lemma 3: For P X z 6 V{X x Z) and i? z e P(-Z), we 
have 

H S J^(P XZ \R Z ) >H*(P xz \Rz). 

Proof: See Appendix iBl ■ 

Lemma 4: For Pxz £ P(<f x Z), we have 

H £ min (Pxz\Pz) < Hi+t(P xz \P z ) - logC 

for any < ( < 1 - e. 

Proof: See Appendix [C] ■ 
From Theorem Q] Lemma [3] and Lemma [4] we have the 
following. 

Theorem 2: For any < r\ < e and < ( < 1 — e, we 
have 

max H £ s - n {Pxz\Rz) + log V - 1 

< t(Pxz,e) 

< Ht C (Pxz\Pz) -logC 

Z). Gaussian Approximation 

In this section, we consider the asymptotic setting. By 
applying the Berry-Esseen theorem to Theorem [2] we have 
the following Gaussian approximation of £(P xz ,e). 

Theorem 3: Let 

V{X\Z) := Pxz(x, z) flog 1 - i?(X|Z) X 

V Px\z\x\z) , 

be the dispersion of the conditional log likelihood. Then, we 
have 



t(Fxz,e) = nH(X\Z) + y/nV(X\Z)$-\e) + O(logn), 

where <£>(•) is the cumulative distribution function of the 
standard Gaussian random variable. 

E. Exponential Bound 

In this section, we review the exponential bounds. 
Definition 3: For P xz £ V(X x Z), let 

Hp\Pxz) =\ogJ2 Pz(z) (^2Px\z(x\z)^J . 

We have the following. 

Theorem 4 ($i?$): For any < p < |, we have 

E F [d(F\Pxz)} < ^\S\Pe^ Pxz \ 

Definition 4: For 6 > 0, Pxz £ P(^x2), and i? z G 
P(Z), let 



H 1+e (Pxz\Rz) :=-±log£i? z (*) ( PX r Z ^ ) Z) ) 



i+e 



be the conditional Renyi entropy of order 1 + 6 relative to Rz- 
For 8 = 0, we define 

H X {P XZ \R Z ) := limif 1+e (Fxz|i?z) 

= H(X\Z) - D(P Z \\R Z ). 



By using Jensen's inequality and by setting p 
have 



1+6' 



<f>(p\P 



xz 



< 



1 



-H 1+e {Pxz\Pz) 



Thus, we have the following slightly looser bound. 
Corollary 3: For < 9 < 1, we have 

E F [d(F\P X z)} < ^\S\^e-^ Hl+e(PxzlPz) . 

From Theorem 2] and Corollary [3] we have the following. 

Theorem 5: We have 



i{Pxz,e) 



-<£(p|Pxz)+log(2e/3) 



(2) 



> sup 

0<p<| 

. mn 0H 1+e (P X z\Pz) + (1 + g)log(2e/3) ... 
o<e<i " 

III. Hybrid Bound 

In this section, we derive another bound from the leftover 
hash lemma (Lemma Q]). A basic idea is to use the smoothing 
in a similar manner as in the derivation of Theorem [4] 
However, we do not use the large deviation bound. 

Theorem 6: For any < rj < e, we have 

t(Pxz,e) 

> max max[6H 1+6 {Pxz\Rz) 
o<e<i r z 

+ (1 - 0)H*-*(P xz \Rz)] + log V - 1. (4) 
Proof: We define the smoothed probability 

P xz (x, z) = P X z{x, z)l 

From Lemma Q] we have 

E F [d(F\P xz )] 

._- ,/|5|e-^(Px Z |fl 2 



(5) 



\ 



1^1 E 



< 



\ 



Pxzj^z) 
Rz(z) 6 



i+e 



-(1-0)7- 



e -effi + e(-Pxz|flz)-(l-e)r_ 



By the triangular inequality, we have 

E F \d(F\P xz )] 



< 2d{P xz ,Pxz) + ^\S\e- eH ^(Pxz\Rz)-(i-e)r 
D / , Pxz(x,z) 



5| e -eHi +9 (Pxz|-Rz)-(l-e)r_ 



Thus, by setting r = v {Pxz\Rz) and by taking \S\ so 
that 

i^/|5|e- 9ffl +e( Pxz l i?z )-( 1 - e ) r < 77, 

we have the statement of the theorem. ■ 
Note that the bound in Theorem [6] interpolates the lower 
bound in Theorem [2] and the bound in Eq. © of Theorem [5] 
More specifically, when the supremum in Eq. (01 is achieved 
by 9 = 0, then the bound in Eq. (0]l reduces to the bound in 
Theorem [2] To derive the bound in Eq. (0), we need some 
large deviation calculation. By using Markov's inequality, we 
have 



, , Pxz(x,z) 



Rz{z) 

< exp{er-0H 1+ g(P xz \Rz)} 



Thus, we have 

H £ s - 7 \Pxz\Rz) > H 1+6 {Pxz\Rz) + ^log(e - 77). (6) 

By setting rj = |, i2z = Pz, and by substituting Eq. (|6]l into 
Eq. ©, we have the bound in Eq. (f3)- 

In ifTTl . the optimal choice of Rz was shown to be 



Rz(z) 



{Y. x Pxz{x,zf 



(7) 



Remark 2: To derive the bound in Eq. (fJJ, we need to 
use more complicated bounding. To derive a non-asymptotic 
bound that subsume the bound in Eq. ©, we need to intro- 
duce more artificial quantities instead of Hi + g(Pxz\Rz) and 
Hr n (Pxz\Rz)- 

IV. Numerical Calculation 

In this section, we consider the i.i.d. setting. We consider 
the case such that Z is obtained from X throughout BSC, i.e., 



1 — 9 1 
Pxz{x,x) = — — , Pxz{x,x + 1) = -. 



(8) 



In this case, since Pz is the uniform distribution on {0, 1}, 
from Eq. (|7), the optimal choice of Rz is Rz = Pz- We have 



P 



l z {-\ogP^ z {x n \z n )<r) 
r + n log(l — q) 



B \n,q, 



log 



1-9 



where B(n, q, k) is the cumulative density function of the 
binomial trial. Thus, the lower and upper bounds in Theorem 
[2] can be described as 

4,iow(e) <t{Pxz,z) < We), 



where 

4,iow(e) = R- 1 (n, q, s- i]) x log 

- nlog(l - q) 
4u P (e) = B _1 (n,g,£ + C) x log 

- nlog(l - q) 



1 



q 

+ log4?7 2 
1-g 

- logC 



1 (9) 



(10) 



For the distribution of the form in Eq. (O, the bound in 
Eqs. (O and (f3]) coincide. We have 



1 



\i+e\ 



H 1+e (Pxz\Pz) = -glog(q i+0 + (l- q y , 
Thus, the bounds in Theorem |5] can be described as 

e(P% z ,e)>l e , low (e), 

where 



^c,low 



sup • 

o<e<i 



^nlog (q 1+e + {l-q) 1+e ) 



■ log(2e/3) - 1 



(11) 



Similarly, the bound in Theorem [6] can be described as 

l{P$z,e) >4,iow(e), 



where 



max 

o<e<i 



-nlog + (1 - g) 1+e ) + (1 - . 



B 1 (n, <?, e - 77) x log 



1-9 



- nlog(l - g) 



log4?7 2 



1. 



(12) 



For e = 10 -10 and q = 0.11, we plot 4,iow(e), 4,up(e)> 
■^e,iow(£), ^h,iow(£)> an d Gaussian approximation derived by 
Theorem [3] in Fig. [T] where we set 77 = ( = | . From the 
figure, we can find that the exponential bound is better than 
the min-entropy bound up to about n = 10000. The hybrid 
bound is better than both the exponential bound and the min- 
entropy bound. The Gaussian approximation overestimate the 
lower bounds, but it is sandwiched by the lower bounds and 
the upper bound. 

In Figs. [2] [3] and |4j the bounds are compared from a different 
perspective, i.e., for fixed n and varying e. From the figures, 
we can find that the exponential bound and the hybrid bound 
become much better than the min-entropy bound as e becomes 
small. When e is rather large for n, the min-entropy bound is 
better than the exponential bound. The hybrid bound is better 
than both the exponential bound and the min-entropy bound 
for whole ranges of e. 

V. Conclusions 

In this paper, we have compared the exponential bound 
and the min-entropy bound. It turned out that the exponential 
bound is better than the min-entropy bound when e is rather 
small for n. When e is rather large for n, the min-entropy 
bound is better than the exponential bound. We also presented 




Fig. 1. A comparison among the bounds for e = 10 and q = 0.11. 
The blue curve is the min-entropy bound £ a i ow (e). The black curve is the 
exponential bound £ e ^ ov ,(s). The red curve is the hybrid bound £^ i ow (e). 
The dashed pink curve is the Gaussian approximation. The green curve is the 
upper bound £ BiUp (e). 
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Fig. 2. A comparison among the bounds for n = 1000 and q = 0.11. 
The blue curve is the min-entropy bound f Sj i ow (e). The black curve is the 
exponential bound £ e i ow (e). The red curve is the hybrid bound £ h i ow (e). 
The green curve is the upper bound £s,up(e). 



the hybrid bound that interpolates the exponential bound and 
the min-entropy bound. 

For a future research agenda, it is important to extend the 
results in this paper to the quantum setting or other information 
theoretic security tasks such as the wire-tap channel. 

Appendix 

A. Proof of Lemma [2] 

Let P S z € B e (P S z) be such that 

Then, we define 

Pxz{x,z) 




-'«.?io( £ ) 



Fig. 3. A comparison among the bounds for n = 10000 and q = 0.11. 
The blue curve is the min-entropy bound £ a i ow (£)- The black curve is the 
exponential bound £ e i ow (e). The red curve is the hybrid bound £^ i ow (£)- 
The green curve is the upper bound ^ s ,up(e). 
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Fig. 4. A comparison among the bounds for n = 100000 and q = 0.11. 
The blue curve is the min-entropy bound £ a i ow (e). The black curve is the 
exponential bound £ e i ow (e). The red curve is the hybrid bound £ h i ow (e). 
The green curve is the upper bound £ B ,up(£)- 



Then, we have 

d{Pxz,Pxz) 

= -j^\ p xz{x,z) - P X z(x,z)\ 

X,Z 



= \^\ p sz{s,z) -P sz (s,z)\ 



S,Z 

= d(P sz ,Psz) 
< s. 

Thus, we have Pxz €E B e (Pxz)- Furthermore, by the con- 
struction of Pxz, we have Pxz(x,z) < Psz(f ( x )> z ) f° r 
every (x,z). Thus, we have 

H e min {P sz \Rz) = H min (Psz\Rz) 

< H min (P X z\Rz) 

< H min( P Xz\Rz)- 



Pxz(x,z)=P sz (f{x),z) 



Psz(f(x),z) 



B. Proof of Lemma \3\ 

Let r = HI(P X z\Rz)- Then, let 



Pxz{x, z) = P X z(x, z)l 
Then, we have 



, Pxz(x, z) 
log 5 -, s > r 
Rz{z) 



d(Pxz,Pxz) < -=. 
Thus, we have Pxz S B e / 2 . Thus, we have 

H E J? n {P X z\Rz) > H min {P xz \Rz) 

> r 

= H £ s {P X z\Rz). 

C. Proof of Lemma [4] 

Let r and Pxz be such that r = H min (Pxz\Pz) 
H^ ia (P X z\Pz)- For arbitrary fixed S > 0, let 

T=Ux,z): Px ^ X \ z) <e-r+> 



Pz{z) 



and 



T z = {x: (x, z) 6 T} . 
Then, we have \T^\ < e r ~ s . Furthermore, we have 

d(P X z,Pxz) > Pxz(T c ) - Pxz(T c ) 

= Pxz(T c )-^2 E p xz(x,z) 

z 167; 

> Pxz(T c )-J2 E e ~ rp z(z) 

z x£T z € 

> Pxz(T c )-e- s . 

Thus, we have 

r — S 

< sup{/:P x ^{-log^|^<r'}< £ + e^ 

= Ht e ' S (Pxz\Pz), 
which implies 

H e min (Pxz\Pz) < m+^ S {P XZ \Pz) + 5 
= H^(Pxz\Pz) -logC 
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